The present invention relates to computer programs used by educators and employers to write tests, analyze and track results.
In particular, the invention is a method in the form of a computer program that facilitates the creation of tests having one or more assessment items, i.e. questions, used for analyzing training, a program of study, job, job performance, or performance criteria of one or more individuals that uses a comprehensive coding system and method for tracking learner successes and allows for integrated assessments using a taxonomy system with taxonomies and codes and/or reference items in relation to each test assessment item.
Several major trends have been and continue to emerge in the education and employment assessment fields. In particular, there has been an increasing emphasis on criterion referenced tests. Criterion referenced tests are those tests having questions, or assessment items, designed to demonstrate a specific ability. Criterion reference testing developed from a recognition that past performance is sometimes the best predictor of future performance. Criterion reference testing may supplement, augment or replace aptitude tests.
For example, a student in a photography class might be asked as one question on a test "What is the difference in light intensity as the light source is moved towards or away from the subject." The answer, pertaining to the inverse square law, would relate to the criteria, that "Student can apply the inverse square law using a single light source." In the employment domain, the criteria for a test item might be "Employee can set up a slicing machine to slice meat to a predetermined thickness." Each statement is a criterion that is an ability.
These types of tests are distinguished from achievement tests in that, contrary to achievement tests that compare individuals achievements in a certain subject area, they are a more absolute measurement of the individual's mastery of a course compentency, i.e., the criterion reference approach. The present invention codes competencies for specific criterion.
Criterion referenced testing becomes significant also as the Americans with Disabilities Act and other similar state, federal and regulatory bodies of law, are followed by employers to assure compliance. In particular, insofar as the ADA is concerned, applicants with certain pre-existing conditions must be fairly screened in the job interview process and in the advancement process. The need for a computer program that allows the individual employer to create assessment items in advance of the job interview, each of which assessment item is integrated with an objective criterion or ability required, such as a competency code and taxonomies, discussed later, allows the ability to more uniformly and fairly examine applicants. Each job, thus, has a certain number of criteria that needs to be met, and establishing these criteria, examining the same and integrating the responses and analyzing the same with competency codes and taxonomies, discussed later, significantly improve the probability of a fair, non-discriminatory treatment of applicants and employees. Job descriptions are prepared that identify essential job duties and that specify the required abilities to perform these tests.
Most jobs require certain skills as basic abilities, transferable skills, along with the specific vocational skills that are required of the effective worker.
While the invention is not limited to its use in education and employment, it is significant to note that the two fields increasingly become interrelated as the educational system moves towards testing based on criterion referencing. The prior art lacks an integrated assessment process. The coding system in the invention allows for an integrated assessment by analyzing several taxonomies within each test item and tracking learner successes with respect to each taxonomy to discover the strengths and/or weaknesses of the learner in that area and correlate them to task competencies and/or duty competencies. While computer programs are available to assist teachers to prepare tests, no program provides a method for creating comprehensive criterion referenced testing using a taxonomy system, competency codes, task importance, skill levels, and references to source information. None has offered any means to assess or analyze transferable skills that are taught, practiced, and acquired in a course of study, training program or job while assessing the referenced criterion at the assessment level.
It is important that educators and employers be able to create their own tests of this kind in the field according to their needs and objectives. Known existing programs for creating tests are very limited, non-comprehensive and primarily allow for preparing the individual assessment items and recording each in their own record in a database only with no comprehensive taxonomy system using multiple taxonomies, or ability to analyze the quality of the test within the context of such taxonomy system and using codes, nor any ability to meaningfully track a testee's responses within the comprehensive taxonomy and code system. The invention also performs this analysis according to each assessment item.
The taxonomy system in the invention is comprised of taxonomies that together make up the broad range of preferred behaviors that are intended to be assessed, such as working with data, working with people, working with things, reasoning, math, language, etc. Another taxonomy system may instead be cognition, psychomotor, affective, working with computers. In each of these the testee may show different behaviors. The invention then tracks and analyzes the results according to the taxonomies and other codes with each question or assessment item to provide an analysis or profile data within a true criterion referenced framework.
The inventive process also includes a reference for each assessment item indicating a reference source of material on which the question is based, i.e., the reference source for learning the subject material. In an employment setting it might reference a particular job analysis previously done. This also allows one then to determine whether what is being assessed has been taught. For example, if a specific item of reference material is not shown in a list of references, then that particular reference item may not be being taught, or covered in the course.
The process also allows for the test creator to input for each assessment item a task importance code for the particular competency being assessed, in order to provide a rating apart from the skill required to successfully complete the test item that rates the importance of the duty that the test item is qualifying.
Moreover, the process allows for a special instruction to be inputted, restored and retrieved with each test item. The special instruction remains with the test item whenever the test item is retrieved from the database, in essence coupled to the test item. The special instruction provides a list of tools or other materials that are to be used as auxiliary materials for that particular test item, by test item number. This allows the course/test matter to print as a cover to the test the auxiliary materials needed to take the test.
The process also allows for a task description statement step to be inputted with each assessment item that is essentially a shorter paraphrase of the overall task that the assessment item addresses. There may be several different assessment items dealing with the same task item. Thus the test creator is allowed to create a course of study for that particular task by retrieving all test assessment items for that task. Of course a manager can enter the test items and use the program to search the relevant codes to develop lessons from the details within the test items.
The process also provides for defining and inputting a skill level code for each assessment item.
Still further, the invention allows for inputting an occupational code for each individual assessment item so as to recall assessment items and all particular coupled information, by occupational cluster code or subgroups within the cluster code.
The invention also allows the merging or splitting of custom test files for a particular individualized instruction, i.e., test creator manually or individually selecting the required codes for the particular required individualized instruction. In particular, the computer program includes the steps of automatically creating individualized assessment and individualized instruction in areas in which a less adequate assessment is achieved.
The inventive process also allows for individual course/job competency files to be created and maintained on an ongoing basis to record competencies from many tests. Moreover, the assessment item is coded as either multiple choice, essay style (short or long answer) or analytical performance. These are categorized as either passive or active assessments. The individual competency files then are analyzed in the process to track the passive or active assessments for each competency, and their respective responses, to determine in the individual's competency file his or her performance. If an individual shows higher performance in passive (multiple choice) questions than in active (essay), then one can differentiate the individual's specific ability with respect to that competency.
The object of the invention then is to create a method using a computer for creating and comprehensively analyzing in an integrated manner a test and course of study or job performance, with the ability to plan a course of study that instructs and assesses multiple transferable skills within the context of course competencies at the individual test item assessment level.
Other objects and features of the invention and the manner in which the invention achieves its purpose will be appreciated from the foregoing and the following description and the accompanying drawings which exemplify the invention, it being understood that changes may be made in the specific method and apparatus disclosed herein without departing from the essentials of the invention set forth in the appended claims.